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ABSTRACT 

This booklet summarizes findings from data on 
mathematics achievement gathered on the 1992 National Assessment of 
Educational Progress (NAEP) for grades 4, 8, and 12. Approximately 
26,000 4th, 8th, and 12th graders in 1,500 public and private schools 
participated in the national assessment. Information included in the 
report are: trends in achievement between 1990 and 1992, distribution 
of overall mathematics proficiency organized by state for grades 4, 
8, and 12, performance on and examples of constructed-response 
questions, national results for demographic subgroups on two regular 
cons tructed-response tasks and an extended-response task, example of 
an extended-response task with scoring guide and sample responses, 
the school context for learning mathematics, trends and trouble spots 
in mathematics instruction, school effectiveness, background 
questionnaires, and procedures and methods. Findings for the 
performance of students on extended constructed-response questions 
include: (1) Approximately one-third to two-thirds of the students 
provided incorrect responses; (2) Substantial percentages of students 
left their papers blank; (3) Most students who did seem to understand 
the problems had difficulty explaining their work; and (4) From 1 to 
16 percent of the students provided extended responses to each one of 
the tasks. On regular constructed-response questions, the average 
percentage correct by grade level was 427. for grade 4, 53V. for grade 
8, and 407. for grade 12. (MKR) 
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What is The Nation's Report Card ? 



THE NATION'S REPORT CARD, the National Assessment of 
Educational Progress (NAEP), is the only nationally representative 
and continuing assessment of what America's students know and can 
do in various subject areas. Since 1969, assessments have been 
conducted periodically in reading, mathematics, science, writing, 
history /geography, and other fields. By making objective information 
on student performance available to policymakers at the national, 
State, and local levels, NAEP is an integral part of our nation's 
evaluation of the condition and progress of education. Only 
information related to academic achievement is collected under this 
program. NAEP guarantees the privacy of individual students and 
their families. 

NAEP is a congressionally mandated project of the National Center 
for Education Statistics, the U.S. Department of Education. The 
Commissioner of Education Statistics is responsible, by law, for 
carrying out the NAEP project through competitive awards to 



qualified organizations. NAEP reports directly to the Commissi >ner, 
who is also responsible for providing continuing reviews, including 
validation studies and solicitation of public comment, on NAEP's 
conduct and usefulness. 

In 1988, Congress created the National Assessment Governing Board 
(NAGB) to formulate policy guidelines for NAEP. The board is 
responsible for selecting the subject areas to be assessed, which may 
include adding to those specified by Congress; identifying appropriate 
achievement goals for each age and grade; developing assessment 
objectives; developing test specifications; designing the assessment 
methodology; developing guidelines and standards for data analysis 
and for reporting and disseminating results; developing standards 
and procedures for intei ate, regional, and national comparisons; 
improving the form and use of the National Assessment; and ensuring 
that all items selected for use in the National Assessment are free from 
racial, cultural, gender, or regional bias. 
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A Synthesis of Findings from 
NAEP's 1992 Mathematics Assessment 



The results from NAEP's 1992 mathematics assessment indicated 
that student performance is improving nationally and in some 
states, but that a considerable challenge remains. The proportions 
of students demonstrating success with more complex tasks 
continued to be low, particularly for those subpopulations of 
students historically considered to be "at risk. " 



Trends in Achievement Between 1990 and 1992 

For the nation, there were statistically significant increases in average 
mathematics proficiency between 1990 and 1992 for fourth-, eighth-, and 
twelfth-grade students. 1 
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Trends in Average Mathematics Proficiency, 
Grades 4, 8, and 12 





GRADE 4 


GRADE 8 


GRADE 12 


Assessment 
Years 


Average 
Proficiency 


Average 
Proficiency 


Average 
Proficiency 


1992 


21 8> 


268> 


299> 


1990 


213 


263 


294 



value for 1992 was significantly higher than the value for 1990 at about the 95 percent confidence 
L Average proficiency was estimated using a scale of 0 to 500. 



1 At each grade NAEP's nationally representative samples included students attending public and 
private schools. ° 
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As presented in Figure 1, 18 of 37 states and territories that participated in 
the grade 8 Trial State Assessment Program in both 1990 ard 1992 showed 
significantly increased average mathematics proficiency for their public- 
school students. 2 



Figure 1 

The NAEP Trial State Assessment - Comparisons 

of Overall Mathematics Proficiency at Grade 8, 1992 vs. 1990 




The increases in mathematics proficiency between 1990 and 1992 for the 
nation And in many states did little to alter the relative standings of the 
various demographic groups. 

• Average performance increased for White students at all three 
grades. The only other statistically significant gains in average 
mathematics proficiency by racial/ethnic group were found for 
Black and Hispanic students at grade 12. In 1992, Asian/Pacific 
Islander and White students had higher average mathematics 
proficiency than did Black students, with American Indian and 
Hispanic students performing somewhere in between. 



2 The NAEP 1990 and 1992 Trial State Assessment Programs included only students attending 
public schools. 



• Average mathematics proficiency for both males and females 
increased at all three grades. In 1992, gender differences were not 
large, but males tended to outperform females students at grade 12. 

• The national increase at grade 4 in overall average mathematics 
proficiency was exhibited in each region except the West. At grade 8, 
there were gains in the Central and West regions, and at grade 12 
students in the Southeast showed improvement in average 
proficiency. The Southeast however, continued to trail behind 

the Northeast, Central, and West at all three grades assessed. 

• Students attending schools in the top one-third of the performance 
distribution showed increased mathematics proficiency at all three 
grades. The only gain for students attending schools in the bottom 
one-third of the distribution occurred at grade 12. 

• As shown in Figures 2 and 3, there was considerable variation in 
performance within a 'cross participating stales and territories. 
Based on sophisticate* i-ests of statistical significance, the states with 
the highest average mathematics proficiency at grade 4 included 
Maine, Iowa, New Hampshire, Wisconsin, North Dakota, 
Minnesota, New Jersey, Connecticut, Massachusetts, and Nebraska. 
At grade 8, the top-performing states included Iowa, North Dakota, 
Minnes ta, Maine, New Hampshire, Wisconsin, and Nebraska. 

Mathematical Problem Solving 

As part of the overall education reform initiative, there has been a 
concerted effort to improve mathematics education in the United States 
by emphasizing problem solving and application in real-life settings, rather 
than rote memorization. Beginning in the mid-1980s, the National Council 
of Teachers of Mathematics (NCTM) worked to develop Curriculum and 
Evaluation Standards for School Mathematics. Published in 1989, The NCTM 
Standards emphasize more balanced and dynamic auricular goals where 
students "do" mathematics — actively exploring, constructing, and 
justifying their ideas as they interact and work to solve problems. 

The Mathematical Sciences Education Board (MSEB) also has 
developed a number of publications supporting the need for such reform, 
stressing the importance of active learning for students, and developing 
prototypes for assessing mathematics performance in ways that support 
instructional goals. 
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Figure 2 

Distribution of Overall Mathematics Proficiency 
Organized by Average Proficiency, 1992 Grade 4 
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The center darkest box Indicates a simultaneous confidence interval around the 
average mathematics proficiency for the Mate based on the Bonferroni procedure 
for multiple comparisons. Center boxes that do not overlap indicate significant 
differences between states in average mathematics proficiency. The darker shaded 
boxes.indicate the ranges between the 25th and 75th percentiles of the mathematicr 
proficiency distribution, and the lighter shaded boxes the ranges between the 5th 
to 25th percentiles and the 75th to 95th percentiles of the distribution. 
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Figure 3 

Distribution of Overall Mathematics Proficiency 
Organized by Average Proficiency, 1992 Grade 8 
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The center darkest box indicates a simultaneous confidence interval around the 
average mathematics proficiency for the state based on the Bonferroni procedure 
for multiple comparisons. Center boxes that do not overlap indicate significant 
differences between states In average mathematics proficiency. The darker shaded 
boxes indicate the ranges between the 25th and 75th percentiles of the mathematics 
proficiency distribution, and the lighter shaded boxes the ranges between the 5th 
to 25th percentiles and the 75th to 95th percentiles of the distribution. 
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There is agreement by the NCTM, MSEB, and a number of national 
organizations, including The National Council on Education Standards and 
Testing, that the potential for educational improvement is enhanced if 
reform is systemic, simultaneously involving such arers as curriculum, 
instruction, assessment, and professional development. To reinforce reform 
efforts, assessments should embody the new instructional goals by 
providing thoughtful problem-solving situations and opportunities for 
students to explain their approaches. 

In NAEP's 1992 mathematics assessment, about one-third of the 
questions and approximately one-half of the students' response time were 
devoted to questions asking students to construct their own responses. 
These questions were classified as regular constructed-response and 
extended constructed-response tasks. Regular constructed-response 
questions required students to provide a short answer giving a solution to 
the problem posed. The extended-response tasks were a new feature of the 
1992 assessment. For these questions, students were allowed at least five 
minutes to complete tasks that required them to demonstrate — by writing, 
by giving examples, or by drawing diagrams — their mathematical 
reasoning and problem-solving abilities. For some of the constructed- 
response questions, NAEP provided students with protractors /rulers, 
calculators, or "manipulable" geometric shapes. 

The procedures employed in constructing and scoring extended- 
response tasks showed that they could be successfully included in a large- 
scale national assessment and that they significantly contributed to 
understanding student proficiency in mathematics at each of the three 
grades assessed. 

In general, the analysis of student papers showed that most made a 
conscientious effort to respond, but the performances exhibited left much to 
be desired. For the nation and across the states, there was a lower level of 
performance on both regular and extended constructed-response questions 
than on the multiple-choice items contained in the 1992 NAEP mathematics 
assessment. A summary of student performance on the constructed- 
response questions follows as do examples of these questions. 

Performance on Constructed-Response Questions 

On regular constructed-response questions, which required only a short 
constructed answer, the average percentage correct by grade level was 
42 percent for grade 4, 53 percent for grade 8, and 40 percent for grade 12. 
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Similar performance was noted across the participating states and territories, 
with the average percentage correct ranging from 27 to 51 percent at 
grade 4 and from 30 to 63 percent at grade 8. 

On extended constructed-response tasks, which required students to 
solve problems requiring a greater depth c* understanding and then 
explain, at some length, specific features of their solutions, the average 
percentage of students producing satisfactory or better responses was 
16 percent at grade 4, 8 percent at grade 8, and 9 percent at grade 12. Similar 
performance was noted across the participating states and territories, with 
the average percentage providing satisfactory or better responses ranging 
from 7 to 22 percent for grade 4 and from 0 to 13 percent for grade 8. 

• Approximately one-third to two-thirds of the students provided 
incorrect responses to these extended questions, indicating little 
evidence of understanding the mathematics concepts involved or 
even the question being asked. 

• Substantial percentages of students, sometimes as many as one-fifth, 
simply left their papers blank. 

• Most students who did seem to understand the problems had 
difficulty explaining their work. 

• It is encouraging, however, that some students — from 1 to 

16 percent — provided extended responses to each one of the tasks. 

Examples of Short Constructed-Response Questions 

The follow ing questions were given to eighth graders as part of the national 
and state assessments, and to twelfth graders as part of the national 
assessment. (State assessments were not conducted at grade 12.) Students 
were provided with hand-held calculators to use in solving the two short 
constructed-response questions. 

To calculate the number of whole packages of paper (reams) that 
Raymond would have to purchase, students were required to calculate the 
number of pages needed (1792) and round that number up to the nearest 
multiple of 500 (2000). For that new number, they then had to determine 
what multiple it was of 500 and respond that Raymond needed to purchase 
4 packages of paper, "ifty-two percent of the eighth graders and 72 percent 
of the twelfth graders completed the question correctly. 
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Example 1 

Numbers and Operations 



Overall Percent Correct* 

Grades — 52 (1.4) 
Grade 12 — 72 (1.4) 

Raymond must buy enough paper to print 28 copies of a report that 
contains 64 sheets of paper. Paper is only available in packages of 
500 sheets. How many whole packages of paper will he need to buy 
to do the printing? 

Answer: ^4" 

Did you use the calculator on this question? 




•The standard errors of the estimated percentages appear in parentheses. 

In the second example, students were asked to determine the area 
of a trapezoid in squaie inches. They needed to disassemble the given 
information and use the area relationship for a rectangle to determine that 
the altitude of the trapezoid was four units. Then they could proceed to use 
the area formula for a tr?r>ezoid or use the area formula for a triangle to find 
the area of triangle ABE and add that area to that for the rectangle BCDE to 
get 80 square inches. In either approach, the mathematical computations 
were not difficult, but students were required to carry out a series of 
sequential calculations in order to obtain the desired answer. This 
problem was quite difficult for students at both grades 8 and 12. Only 
10 percent of the eighth graders and 23 percent of the twelfth graders 
provided a correct response. 
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Example 2 

Measurement 



Overall Ptretnt Correct* 

Grade 8— 10(0.9) 
Grade 12 — 23 (1.6) 



The area of rectangle BCDE shown above is 60 square inches. If the 
length of AE is 10 inches and the length of ED is 15 inches, what 
is the area of trapezoid ASCD, in square inches ? 



Answer: 



20. 



Did you use the calculator on this question? 
Yes ^) 



The standard errors of the estimated percentages appear in parentheses. 

The national results for demographic subgroups for "Raymond's 
Report'' and "Area of a Trapezoid" for grades 8 and 12 are found in Table 1, 
and the state results for both questions at grade 8 are found in Table 2, In 
general, students had more success solving the problem of "Raymond's 
Report" than they did in finding the "Area of a Trapezoid." For example, 
across the states, from 18 to 71 percent of the eighth graders were able to 
find the correct answer to "Raymond's Report." In contrast, performance 
ranged from 1 to 16 percent correct in finding the trapezoid's area (even 
when the solution could have been determined by adding the area of the 
rectangle to the area of the triangle). In general, the level of correct 
responses suggests that students have little grasp of how to integrate and 
sequence information to solve a problem. 
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Table 1 

National Results for Demographic Subgroups 
for the Regular Constructed-Response Tasks, 
"Raymond's Report 7 ' and "Area of Trapezoid" 

GRADE 8 



Raymond's Report Area of Trapezoid 





Correct 


Incorrect 


No 
Response 


Correct 


Incorrect 


No 
Response 


Nation 


52 (1.4) 


46 (1.4) 


3(0.4) 


10 (0.9) 


81 (1.3) 


9 (0.8) 


Northeast 


58 (3.5) 


38(3.5) 


4(0.7) 


9(1.4) 


80 (1.7) 


11(1.5) 


Southeast 


42 (2.9) 


54 (3.1) 


4(1.1) 


9(1.9) 


82 (2.8) 


9(1.3) 


ueniidt 


61 (3.3) 


39 (3.4) 


0 (0.2) 


10(1.9) 


84(2.1) 


5 (0.8) 


West 


n (1.9) 


49(1.6) 


3(1.0) 


10(2.0) 


79 (3.0) 


11 (2.3) 


White 


62 (1.7) 


36(1.7) 


2 (0.4) 


12(1.2) 


80 (1.4) 


8(0.7) 


Black 


20 (3.2) 


74 (3.5) 


6(1.1) 


• 2(1.1) 


84 (3.8) 


14 (3.3) 


Hispanic 


30 (3.7) 


65 (3.8) 


5(1.4) 


3(1.3) 


88 (2.3) 


9(1.9) 


Male 


51 (2.3) 


46 (2.4) 


3(0.6) 


10(1.4) 


79 (2.0) 


12(1.4) 


Female 


52 (1.9) 


45(1.9) 


2(0.5) 


9(1.2) 


84(1.5) 


7 (0.9) 


Advantaged Urban 


66 (5.3) 


34 (5.3) 


0 (0.0) 


19(3.8) 


70 (4.0) 


11 (4.9) 


Disadvantaged Urban 


25 (4.5) 


68 (4.8) 


6(1.7) 


4(1.3) 


82 (2.8) 


14(3.1) 


Extreme Rural 


55 (8.2) 


45 (8.1) 


0 (0.4) 


9 (3.3) 


83 (4.8) 


8(2.7) 


Other 


52 (1.6) 


44(1.6) 


3 (0.6) 


9(1.2) 


83 (1.5) 


9 (0.7) 


Public 


50(1.5) 


47(1.5) 


3 (0.5) 


9(1.0) 


82 (1.4) 


9 (0.9) 


Catholic and Other Private 


62 (3.2) 


35 (3.2) 


3 (0.6) 


12 (2.0) 


78 (2.5) 


9(1.6) 








GRADE 12 








Raymond's 


Report 


Area of Trapezoid 








No 






No 




Correct 


incorrect 


Response 


Correct 


Incorrect 


Response 


Nation 


72(1.4) 


25(1.4) 


2(0.5) 


23 (1.6) 


67 (1.6) 


10(0.9) 


Northeast 


75 (2.2) 


23 (2.3) 


2 (0.9) 


26 (2.8) 


64 (3.4) 


10(1.6) 


Southeast 


68 (2.8) 


28 (2.7) 


3(1.1) 


16(2.3) 


77 (2.2) 


7(1.5) 


Central 


78 (1.9) 


21 (2.2) 


2 (0.9) 


26 (3.2) 


64(2.1) 


10(2.4) 


West 


69 (3.5) 


29 (3.7) 


2 (0.6) 


24 (4.2) 


65 (4.3) 


11(1.4) 


White 


78 (1.3) 


20 (1.3) 


1 (0.3) 


27 (2.1) 


63 (1.8) 


10(1.1) 


Black 


51 (4.2) 


44 (4.7) 


5(1.5) 


8(1.9) 


81 (3.0) 


12(2.6) 


Hispanic 


62 (5.9) 


34 (6.2) 


4(2.4) 


14 (2.3) 


80 (2.5) 


6(1.8) 


Male 


74 (2.0) 


23 (2.0) 


4 (0.9) 


24(1.7) 


65 (1.8) 


10(1.2) 


Female 


71 (1.9) 


28 (1.9) 


1 (0.3) 


22 (2.4) 


68 (2.2) 


9(1.3) 


Advantaged Urban 


79 (3.6) 


21 (3.7) 


0 (0.2) 


46 (4.8) 


50 (4.6) 


4(1.4) 


Disadvantaged Urban 1 


62 (3.4) 


33 (3.8) 


5(1.4) 


10(2.1) 


79 (2.7) 


11 (2.3) 


Extreme Rural 


73 (3.8) 


25 (3.5) 


2(1.0) 


25 (4.0) 


68 (2.7) 


8(2.4) 


Other 


73 (1.5) 


25 (1.6) 


2 (0.6) 


21 (2.6) 


68 (1.9) 


11 (1.0) 


Public 


72 (1.5) 


26(1.5) 


2(0.5) 


21 (1.9) 


70 (1.8) 


10(0.9) 


Catholic and Other Private 


79 (2.4) 


20 (2.4) 


2 (0.8) 


42 (3.4) 


49 (3.5) 


9(1.8) 



The standard errors of the estimated percentages appear in parentheses. It can oe said with about 
95 percent certainty that for each population of interest, the value for the whole population is within plus 
or minus two standard errors of the estimate for the sample. In comparing two estimates, one must use 
the standard error of the difference (see Appendix for details). When the proportion of students is either 
0 percent of 100 percent, the standard error is inestimable. However, percentages 99.5 percent and greater 
were rounded to 100 percent and percentages 0.5 percent or less ere rounded to 0 percent. Percentages 
may not total 100 percent due to rounding error. 

SOURCE: National Assessment of Educational Progress (NAEP), 1992 Mathematics Assessment 
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Table 2 

Percentages of Correct Responses 

to Regular Constructed-Response Questions "Raymond's Report" 
and "Area of Trapezoid," with Calculator Available 

GRADE 8 -1992 



Raymond's Report Area of Trapezoid 

No No 



rtlDllb dCnUUlS 


uorreci 


incorrect 


nBsponso 


torrect 


incorrect 


nosponso 


Nation 


5fl M 51 


47 (1 51 

HV \\.0) 


3 fn 5i 

O \\J.o) 


q m m 
y u .uj 


fl9 M 41 
Ot 1 1 .HJ 


Q (fl Ql 


Mnrthftact 
ntUIUIoaM 


5fi td 11 

\Jv IH. 1 1 


3Q td 1 1 

OJJ IH. 1 J 


3 (ft fll 

O yJ'O) 


7 M Ql 


fll (9 91 


19 '1 71 


OUUlllcaal 


3fl 19 51 

OO 


5fl (9 Ql 

30 


d l1 91 


Q (9 1 1 


flo n n\ 


Q /1 41 

y \\A) 


Centra! 


60 (3.7) 


39 (3.7) 


0 (0.2 


10 (2.3) 


85 (2.3 


5(1.0) 


West 

A! a ha ma 


47 (2.0) 


49 (1.7) 


4(1.1) 


9 (2.0) 


79 (3.2) 


11 (2-5) 


41 (9 3) 


5fi (9 31 


3 (fl 71 


4 (fl 71 


flQ M 51 


7 M 31 


AriTnna 

r\\ ItUI la 


53 19 9\ 


44 (9 91 


3 (P. 71 


Q (1 31 
y ^ 1 .OJ 


fl9 r1 fll 


Q /1 91 




43 M Ql 

HO 1 1 


54 (9 01 


3 (fl fll 


5 (fl fll 


flQ M 91 


R (fl Ql 


California 


48 (2.3 


47 (2.2) 


5(1.0 


10(1.3) 


76 (1.5) 


14(1.4) 


Colorado 


56 (1.9) 


41 (1.9) 


3(0.8) 


9(1.4) 


81 (1.5) 


10(1.2) 


Connecticut 


58 (2.2) 


40 (2.3) 


2 (0.6) 


12(1.4) 


81 (1.8) 


7(1.1) 




54 (2 71 


43 (9 31 

HO \£..Of 


9 (n qi 


R M 31 


fl5 M 71 


Q M 31 
y \ 1.0) 


District of Columbia 


30 (2.8) 


62 (2.7) 


fl (1 91 


3 M 11 
0\\.\) 


fl7 ^9 91 


m c\ qi 


Florida 


59 (9 1 1 


44 (2.1) 


4 M ni 


r (n qi 


fl5 M Rl 
Oo ^ 1 .0^ 


Q M 41 


Georgia 


45 (2.3) 


52 (2!6) 


3 (0.9) 


5(1.0) 


90(1.3) 


4 (0.8) 


Hawaii 


48 (2.1) 


48 (2.1) 


4(0.8) 


8(1.2) 


79 (1.9) 


13(1.5) 


Idaho 


58 (1.7) 


39 (1.6) 


2 (0.6) 


13(1.6) 


79 (1.8) 


8(1.0) 


Indiana 

ii iuioi in 


55 (9 01 

OO \C..\Jf 


43 (9 01 

HO 


9 (fl 71 


Q (1 31 
y \ \.o) 


fld M Rl 

OH \ 1 .0^ 


7 (1 11 


Iowa 


71 (9 9\ 


9fl (9 31 


9 ((1 51 


13 M 51 


AO M ^\ 
04 ^ 1 -0 J 


o p.uj 


Kpntunkv 

rvui i iu ut\j 


54 (9 31 

OH \C..Of 


43 (9 91 


3 (ft 71 
O \\J.f) 


7 M 11 
t \\.\) 


fl7 M 41 
Of ^ I.HJ 


R (C\ Ql 


Louisiana 


42 (2.5) 


54 (2.4) 


5(1.1) 


4(1.1) 


90 (1.6) 


7(1.5) 


Maine 


67 (2.3) 


32 (2.3) 


2(0.4) 


12(1.5) 


80 (1.9) 


8(1.3) 


Maryland 


54 (2.4) 


42 (2.2) 


4(1.0) 


9(1.5) 


81 (2.0) 


10(1.4) 


M a<><iprh u^tte 


59 (2.6) 


3Q (9 51 

03 ^£.OJ 


3 (fl fll 


Q M 1 1 


fl9 M 41 

0£ ^ 1 .4^ 




Mirhioan 

if iiuuiyai i 


55 (9 11 

OO \C 1 J 


43 (9 91 
HO ^.^J 


9 (fl 71 


m m 41 

IU \ 1 .H^ 


fll M R\ 


fl (1 91 


Minnp<>nta 

1*111 II IwOU LQ 




39 f 1 71 


9 (fl Rl 


15 M 71 
10 [\.f) 


7fl M Q\ 
r 0 ^ i .y } 


7 m m 


Mississippi 


35 (2.1) 


61 (2.2) 


4 (0.9) 


3 (0.7) 


89 (1.1) 


8(1 0) 


Missouri 


53 (1.9) 


46 (1.9) 


1 (0.5) 


8(1.2) 


85 (1.6) 


7(1.2) 


Nebraska 


58 (2.3) 


41 (2.3) 


2 (0.6) 


12(1.5) 


83 (1.8) 


4 (0.8) 


New Hampshire 


63 (2.1) 


36 (2.1) 


1 (0.5) 


13(1.5) 


78 (1.9) 


8(1.3) 


New Jersey 


59 (3.2) 


37 (3.1) 


3(1.0) 


9(1.7) 


84 (2.3) 


7(1.2) 


New Mexico 


46 (2.2) 


52 (2.2) 


2 (0.6) 


7(1.1) 


85 (1.6) 


7(1.2) 


New York 


55 (2.4) 


43 (2.4) 


2 (0.8) 


12(1.7) 


84(1.8) 


4 (0.9) 


North Carolina 


48(1.9) 


50(1.8) 


2 (0.6) 


5 (0.9) 


90(1.2) 


6 (0.8) 


North Dakota 


67 (2.0) 


32 (2.0) 


1 (0.5) 


16(1.8) 


79 (1.9) 


5(1-0) 


Ohio 


58 (2.3) 


41 (2.2) 


1 (0.4) 


8(1.0) 


87 (1.4) 


6(1.0) 


Oklahoma 


59 (2.5) 


39 (2.5) 


2(0.5) 


8(1.2) 


85 (1.4) 


6(1-2) 


Pennsylvania 


56 (2.5) 


41 (2.6) 


2 (0.7) 


10(1.3) 


84 (1.9) 


7(1.4) 


Rhode Island 


54 (2.2) 


45 (2.0) 


2 (0.6) 


6(1.0) 


88 (1.4) 


6 (0.8) 


South Carolina 


44 (2.5) 


54 (2.4) 


3(0.7) 


8(1.1) 


86 (1.4) 


6 (0.9) 


Tennessee 


44 (2.4) 


52 (2.4) 


4(0.7) 


5(1.0) 


88 (1.4) 


7(1.0) 


Texas 


51 (2.3) 


45 (2.2) 


4(0.7) 


9(1.2) 


83 (1.6) 


8(1.2) 


Utah 


61 (1.8) 


38 (1.8) 


1 (0.4) 


10(1.3) 


83 (1.4) 


7(1.1) 


Virginia 


57(2.1) 


42 (2.0) 


2 (0.6) 


8(1.1) 


86 (1.3) 


6(1.1) 


West Virginia 


52 (2.2) 


46 (2.1) 


2 (0.5) 


5 (0.8) 


88 (1.3) 


8(1.1) 


Wisconsin 


65 (1.7) 


34 (1.5) 


1 0.4) 


11 (1-3) 


83 (1.3) 


6 (0.9) 


Wyoming 


61 (2.2) 


37 (2.3) 


2 (0.8) 


9(1.1) 


86 (1.3) 


6 (0.8) 


Territories 














Guam 


29 (2.8) 


64 (2.7) 


7(1.3) 


4(1.0) 


87 (1.8) 


9(1.7) 


Virgin Islands 


18(1.8) 


68 (2.2) 


14 (2.0) 


1 (0-5) 


82 (2.1) 


17(2.2) 



The standard errors of the estimated percentages appear in parentheses. It can be said with 95 percent 
certainty that for each population of interest, the value for the whole population is within plus or minus 
two standard errors of the estimate for the sample. In comparing two estimates, one must use the 
standard error of the difference (see Appendix for detaib). 

SOURCE: National Assessment of Educational Progress (NAEP), 1992 Mathematics Assessment 



ERLC 



16 BEST COPY AVAILABLE 



Example of Extended-Response Task: Radio Stations 

An example of an extended constructed-response task given to eighth 
graders is shown in Example 3. In "Radio Stations," students were asked to 
apply measurement and geometry knowledge to diagram the intersection 
of signal transmissions from two radio stations. 

An important aspect of mathematical power is the need to use logic and 
diagrams to make sense of a situation and to communicate this reasoning. 
However, the results indicate that many students do not recognize that 
diagrams can be effective analytical and communications tools. Even 
though a variety of diagrams or explanations could be used to help explain 
the intersection of the broadcast areas of the two radio stations and no 
particular approach was preferred, only 5 percent of the eighth graders were 
able to read and interpret the question and translate this information to 
develop a labelled model that represented the situation. 

Students with incorrect responses provided no evidence that they were 
able to make sense of the problem, often copying a piece of inf ormation 
from the problem or submitting a meaningless drawing (or both). Forty-five 
percent of the eighth graders nationally provided such responses and 
another 16 percent did riot answer the question at all. 

Although this meant the majority appeared to be essentially at a 
loss as to the nature of this task, about one-third did seem to have some 
understanding of the information presented in relation to the task required. 
Approximately 22 percent received minimal credit and another 13 percent 
received partial credit. The difficulty with these responses was that they were 
incomplete, at best understandable only to those familiar with the problem. 
These sketchy solutions appeared in spite of directions explicitly telling 
students what to diagram and to be sure to label the distances and the part of 
the highway where both stations could be received. 



Example 3 

Extended Constructed-Response Task 

Grade 8 Question: Radio Stations 
The Task 



This question requires you to show your work and explain your reasoning. 
You may use drawings, words, and numbers in your explanation. Your answer 
should be clear enough so that another person could read it and understand 
your thinking. It is important that you show all your work. 



Radio station KMAT in Math City is 200 miles from radio station KGEO in 
Geometry City. Highway 7, a straight road, connects the two cities. 

KMAT broadcasts can be received up to 1 50 miles in all directions from the 
station and KGEO broadcasts can be received up to 125 miles in all directions. 
Radio waves travel from each radio station through the air, as represented below. 



d a ' Wave 
Radio ; ; [ t % \\\\ 

Sution"Trrr Jjjj 



On the next page, draw a diagram that shows the following. 

• Highway 7 

• The location of the two radio stations 

• The part of Highway 7 where both radio stations can be received 

Be sure to label the distances along the highway and the length in miles of 
the part of the highway where both stations can be received. 



15 

18 



Possible Solution 



20C >Mli-6S 




There is a 75-mile part of Highway 7 that is within both broadcast areas. 
It starts 75 miles outside Math City and ends 150 miles outside Math City. 

Students need to assimilate and translate semantic information in order 
to draw a diagram that graphically depicts the location of the radio stations 
and Highway 7 accurately in terms of given boundary conditions. A graphical 
approach to this task should enable students to determine the length of the 
overlapping portion of Highway 7, along which both radio stations can 
be received. Any satisfactory response must clearly illustrate an overlapping 
region, whereas, in addition, any extended response must clearly identify 
the overlap and correctly determine its length to be 75 miles. 
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National Results, Scoring Guide, and Sample Responses 



National Percentage* = 16(1.1) 



I NTO K It I CT 



National Percentage = 45 (1.6) 

Scoring Guide — The work is completely incorrect or irrelevant, or the 
response states, "I don't know/' 

































7 ' 



TTris INCORRECT response does not relate the information given in the problem in 
a manner that conveys either a meaningful problem solving approach or an adequate 
solution. 



*The standard errors of the estimated percentages appear in parentheses. 
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MINI M A I 



National Percentage = 22(1.2) 

Scoring Guide — Diagram with only cities, Highway 7, and 200 miles 
labeled; or a diagram that shows some, but not all, of the given distances: 
125, 150, or 200 miles. Minimal responses do not recognize that the 
common broadcast area is a length along the highway. 




This MINIMAL response correctly depicts two pieces of information (radio stations 
KMAT and KGEO are 200 miles apart and station KGEO can broadcast 125 miles 
and shows rudimentary understanding. It does not show the common broadcast 
area as a length along the highway. 



18 
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National Percentage « 13 (0.9) 

Scoring Guide — Diagram with cities, Highway 7, and 200 miles labeled and 
identification of common broadcast area as a length along (or not on) the 
highway. Two or more of the radio wave distances (250, 125, and 75) are 
insufficiently labeled. 




CR* bd& Vto&r} 

This PARTIAL response indicates considerable understanding of the task relative to 
the given information. The diagram shows the radio stations to be 200 miles apart 
and that KMAT can broadcast 150 miles. Additionlly, the diagram shows a part of 
the highway (from A to B) along which both radio stations can be heard. However, 
the response does not show the broadcast range of station KGEO and does not 
indicate the length of the commo>. >oadcast area. 



19 
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National Percentage = 4 (0.5) 

Scoring Guide — Diagram with cities, Highway 7, 200 miles, and all radu> 
wave distances labeled and identification of common broadcast area on 
Highway 7 as a length. At the same time, omits or incorrectly computes 
length of the highway along which both radio stations can be received. 




This SATISFACTORY diagram shows a good understanding of the problem. 
Although the student correctly labeled the common area along Highway 7 where 
the two stations could be heard, the length in miles of this region was not indicated. 
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23 



National Percentage « l (0,3) 



Scoring Guide — An accurate, well-labeled diagram (as described in the 
"Satisfactory" category) clearly indicating that the portion of Highway 7 
along which both radio stations can be received is 75 miles ir length. 




Tilts is a so/id EXTENDED response. The diagram is accurate and well-labeled. 
Additionally, below the diagram a statement correctly concludes that the length of 
the part of Highway 7 along which both radio stations can be heard is 75 miles. 
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Table 3 

National Results for Demographic Subgroups 
for the Extended-Response Task, "Radio Stations" 

GRADE 6 



No 

Rtponte Incorrect Minimal 



Nation 


16(1.1) 


45 (1.6) 


Northeast 
Southeast 
Central 
West 


15(17) 
18(2.1) 
12(2.2) 
17(2.5) 


42 (3.5) 
50(2.8) 

42 (2.0) 

43 (3.5) 


White 
Black 
Hispanic 


11 (1.2) 
32(4.1) 
26 (2.5) 


40 (2.0) 
55 (4.2) 
58 (2.9) 


Male 
Female 


17(1.2) 
14(1.7) 


46 (2.1) 
43 (2.0) 


Advantaged Urban 
Disadvantaged Urban 
Extreme Rural 
Other 


4(1.7) 
38 (4.6 
15(4.9) 
15(1.5) 


32 (3.2) 
49 (4.7 
39 7.3) 
47 (2.2) 


Public 

Catholic and 
Other Private 


17(1.2) 
8(1.6) 


45(1.8) 
42 (2.7) 



22 (1.2) 

22 (2.0) 
17(1.7) 
26 (2.8) 

23 (2.5) 

26 (1.6) 
8(2.1) 
11 (2.0) 

19(1.8) 
24(1.7} 

30 (3.3) 
7(1.7) 
30 5.6) 
21 (1.4) 

21 (1.4) 
25 (2.3) 



13(0.9) 


4 (0.5) 


15(2.5) 


5(1.2) 


12 1.4 


3 0.8) 


14 (2.4) 


5 1.3 


10(1.2) 


4(1.1) 


16(1.2) 


5 (0.8) 


4 1.6 


1 (0.6) 


4(1.3) 


1 (0.6) 


13(1.3) 


4(0.7) 


12(1.3) 


4 (0.8) 


24 (3.1) 


7(1.8) 


4(1.0) 


2(1.5) 


14 4.8) 


2(1.0 


12(0.9) 


4 (0.7) 


12(1.1) 


4 (0.6) 


18(2.2) 


6(1.3) 





Satisfactory 


Exttnded 


or Bailor 


1 (0.3) 


5(0.6) 




6(1.1) 


o^i 


3 (0.8) 


1 (0-2) 


6(1.3) 


2 (0.9) 


6(1.4) 


2(0.4) 


7(0.9} 


0 (0.0) 


1 (0.6) 


0 (0.0) 


1 (0.6) 


1 (0.3) 


4 (0.8) 


2(0.5) 


6(1.0) 


3(1.0) 


10(1.5) 


0 (0.0) 


2(1.2) 


0 (0.5) 


2(1.2) 


1 (0.4) 


5 (0.8) 


1 (0.3) 


5(0.7) 


1 (0.4) 


7(1.4, 



The standard errors of the estimated percentages appear in parentheses. It can be said with about 95 percent certainty 
that for each population of Interest, the value for the whole population is within plus or minus two standard errors 
of the estimate for the sample. In comparing two estimates, one must use the standard error of the difference (see 
Appendix for details). When the proportion of students is either 0 percent or *100 percent, the standard error is inestimable. 
However, percentages 99.5 percent and greater were rounded to 100 percent and percentages 05 percent or less were 
rounded to 0 percent. Percentages may not t^al 100 percent due to rounding error. 

SOURCE: National Assessment of Educational Progress (NAEP), 1992 Mathematics Assessment 



Across the categories of students by region, race/ethnicity, gender, type 
of community, and type of school, a majority of only one subgroup provided 
at least minimal responses: advantaged urban students. From 32 percent 
(advantaged urban) to 58 percent (Hispanic) of the students by subgroup 
provided meaningless information (see Table 3), 

As shown in Table 4, the percentages of success for public-school eighth 
graders in the jurisdictions participating in the Trial State Assessment 
Program were similar to those for the nation. However, in two states, Iowa 
and Minnesota, at least 10 percent of the students were estimated to have 
provided satisfactory or better diagrams. For five states, Iowa, Minnesota, 
Nebraska, New Hampshire, and North Dakota, the majority of the students 
were estimated to have provided minimal or better responses. 
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Table 4 



Percentages for K iponses 

to Extended-Response Question, "Radio Stations' 4 



GRADE 8 -1992 



Public Schools 



No Satisfactory 
Roponst Incorrect Minimal Partial Satisfactory Extended or Bettor 



Nation 17(1.2 

Northeast 17(2.0' 

Southeast 19 2.4) 

Central 14(2.6) 

West 17(2.8) 

States 

Alabama 22 (2.0) 

Arizona 16(1.7) 

Arkansas 13(1.8) 

California 19(1.7) 

Colorado 11 (1.2 

Connecticut 10(1.2 

Delaware 16 (2.0) 
District of Columbia 30(2.1) 

Florida 22(1.8 

Georola 19(1.7) 

Hawaii 23(1.8) 

Idaho 11 (1.1) 

Indiana 7(1.1) 

iowa 6 (0.9) 

Kentucky 13(1.4) 

Louisiana 24 2.2 
Maine 7 (0.9) 

Maryland 15(1.7) 

Massachusetts 12(1.2 

Michigan 15(1.61 

Minnesota 6(1.1) 

Mississippi 20(1.9) 

Missouri 10 ' 

Nebraska 6 

New Hampshire 8(1.1) 

New Jersey 12(1.6) 

New Mexico 17(1.7) 

New York 15(2.1) 

North Carolina 12(1.4) 

North Dakota 6(1.2) 

Ohio 12(1.3) 

Oklahoma 10(1.4) 

Pennsylvania 12(1.2) 

Rhode Island 10(1.1 

South Carolina 12 1.4 

Tennessee 16(1.5) 

Texas 16(1.9) 

Utah 10(1.4) 

Virginia 13(1.5) 

West Virginia 13(1.5) 

Wisconsin 9(1.9' 

Wyoming 8(1 .2! 

Territories 

Guam 45 (2.4) 

Virgin Islands 47 (3.4) 



l!3 



45(1.8] 

40 (4.2 

52 2.7 

43 2.5 

44 (4.0) 

53 (2.4) 

48 2.3) 
57 (2.6 
44(2.3) 
43(2.0] 
44(2.3' 

51 (2.4; 
57(2.3! 

46 (2.2 

49 2.1 

47 2.3) 
46 (2.2) 

43(2.6 
35 1.8! 

52 2.3' 

54 2.1 ! 
46 2.1 

48 (2.1 1 

45 (2.8) 

46 1.9 

41 (2.3 
57 (2.2 

47 (2.1 
44 (2.8) 

42 (2.1) 

46 (2.3 

51 (2.2; 

44 (2.4 
54(2.2; 
43(2.5! 

47 (2.1) 
46 (2.0 

44 (2.1 

49 (3.5) 

55 (2.1) 
51 (2.4) 

45 (2.2) 

46 (2.5) 

48 (2.5) 
53(2.4) 

43 (1.8) 
45 (2.4) 

40 (2.4) 

47 (3.3) 



21 (1.4 

22 (2.6) 
15(1.8 
25 (3.3) 

23 (2.7) 

14(1.5) 

20 (1.8) 
18(1.8) 
19(2.1) 
23(1.6) 
23 (2.0) 

17(1.5; 
8(1.0! 
19(1.9' 
19(1.6 
15(1.6; 

21 (2.0| 

25 (2.1) 
28 (1.9) 

22 (1.7) 
15(1.4) 

21 (1.9) 

20 (1.8) 

22 (2.4) 

21 (1.9) 
20 (1.9) 
14(1.7) 

23 (1.9) 

23 (2.0) 

24 (1.8) 

22 (2.0) 
19(1.6; 
22 (1.6 



21 (1.8) 
25 (2.0) 



25 (2.1) 
23 (1.6) 
22 (1.7) 
25 (2.9) 

21 (1.7; 

22 (1.7 

21 (1.5) 

20 (1.7) 

22 (1.7) 

21 (1.7) 

23 (1.8) 

22 (1.7) 

10(1.9) 
4(1.1) 



12(1.1* 
16(3.1) 

11(1-4) 
13 (2.9) 
9(1.4) 

8(1.1) 
13(1.6) 

8(1.2) 
13 (2.1) 
16(1.7) 
16(1.7) 

12(1.8) 
3(1.1) 

10(1.3) 
7(1.1) 

11(1.3' 

15(1.2! 

15(1.9) 
21 (2.0 
10(1.1) 
6(1.0) 
18(1.8) 
13(1.6) 

14(1.9) 
12(1.1) 
21 (2.4) 
6(1.2) 
12(1.6) 
20(3.0) 

18(1.5) 
14(1.8) 
11(1.4) 
11(1.5) 
10(1.5) 
18(1.8) 

12(1.8) 
16(1.7) 
14(1.4) 
12(1.6) 
10(1.2) 
10(1.6) 

12(1.4) 
16(1.5) 
12(1.4) 
10(1.2) 
18(1.9) 
18(1.6) 

3(1.0) 
1 (0.7) 



4(0.6 
4(1.5) 

2 0.7) 

4 1.3) 
4(1.2) 

2(0.6) 
2(0.7) 
2(0.6) 
3(0.9) 
5(0.9) 
6(1.0) 

3(0.9) 

1 (0.5 

3 (0.9) 

4 0.9) 
3(0.8) 
5(1.0) 

3(0.7) 
6(0.9) 

2 0.5 

1 0.4 
6 0.9 
3(0.9) 

5 (0.9) 

5 (0.9) 
7(1.0] 

2 (0.6* 

sjo.g; 

6(1.0! 

5(1.0] 
5 0.9 

3 0.7) 
5 (0.9) 
2(0.5) 

5 (0.9) 

5(1.0) 

3 (0.8) 
6(1.1) 
2 (0.8 

2 (0.6) 

1 (0.5) 

3(0.7) 
7(1.2) 

4 (0.8> 

3 (0.6) 
4(1.0) 

6 (0.9) 

2 (0.8) 
0 (0.3) 



1 (0.3) 
1 (0-5) 

0 (0.3) 

1 (0.2) 
3(1.0) 

1 (0.4) 
1 (0.4) 
1 (0.3) 

1 (0.5) 

2 (0.6) 
2(0.4) 

1 (0.5) 
1 (0.5) 
1 (0.3) 
1 (0-5) 

1 (0-4) 

2 (0.6) 

1 (0.6) 
4(1-1) 

1 (0.4) 

0 (0.2) 

2 (0.6) 

1 (0.5) 

2(0.7) 

2 (0.6) 
4(1.0) 

0 (0.2) 
2(0.7) 

1 (0.6) 

3 (0.9) 
1 (0-5) 
1 (0.3) 
2(0.7) 

1 (0-4) 
3(1-2) 

0 (0.2) 

2 (0.8) 

3 (0.7 
2 (0.6) 

1 (0-4) 

0 (0.2) 

3(1.1) 
2(0.4) 

1 0.5) 

0 (0.3) 
4(1.0) 
3(0.7) 

1 (0.5) 
0 (0.0) 



The standard errors of the estimated percentages appear in parentheses. It can be said with 95 percent 
certainty that for each population of interest, the value for the whole population is within plus or minus 
two standard errors of the estimate for the sample. In comparing two estimates, one must use the 
standard error of the difference (see Appendix for details). 

SOURCE: National Assessment of Educational Progress (NAEP), 1992 Mathematics Assessment 
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The School Context for Leading Mathematics 



There were very large performance differences within grade level between 
the achievement of students in the highest-performing one-third of the 
schools compared to the lowest-performing one-third of the schools. These 
are not small or extreme segments of the school population. For example, 
one-third of the twelfth graders in the nation represents more than one 
million students. It is a matter of concern, then, to find that the twelfth 
graders in the lower-performing third of the schools had lower average 
mathematics achievement than the eighth graders in the higher-performing 
one-third of the schools for that grade. 

Because it is considerably more difficult to implement reform in schools 
with iarge numbers of disadvantaged students, 3 it is of further concern to 
find the extent to which economically disadvantaged and minority students 
are concentrated in the lower-performing schools. Very few students in the 
top-performing one-third of schools (0 to 2 percent) were attenau ^ schools 
in disadvantaged urban communities, and these students showed very little 
racial/ethnic diversity (84 to 88 percent White students across the three 
grades assessed by NAEP). At grades 4 and 8, ?bout half the students in the 
\r .^er-performing one-third of the schools were in schools where the 
majority of the students were participants in the subsidized lunch program. 

• Students in the top-performing schools have a greater opportunity to learn 
by virtue of being in school more often and with less transiency. In the 
top-performing schools there was less absenteeism, class cutting, 
tardiness, and transiency. The continuity of instructional approach 
that can be adopted in these schools simply is not possible in the 
lower-performing schools. By grade 12, all but 1 percent of the 
students in bottom-performing schools were in schools where 
principals considered absenteeism at least a minor problem. 

• In general, students in the top-performing schools have considerably more 
support for academic achievement than those in lower-performing schools. 
Considerable research has found schools with positive school 
climates to be more effective than those without any central purpose 



3 O'Day, J.A. & Smith, M.S. "Systemic Reform and Educational Opportunity." In S.H. Furhman, editor, 
Designing Coherent Policy: Improving the System (San Francisco, CA: Jossey-Bass, 1993). 
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Percentages of 12th Graders in Bottom One-Third 
Performing Schools with Negative Ratings about 
Academic Interest and Parental Support 

Negative Student 
Attitudes about 
Achievement 

Negative Ratings 
about Parental 
Support 




or goals, and that such schools can work for even the most 
disadvantaged students. 4 For the twelfth graders in the lowest- 
performing third of schools, 22 percent were in schools receiving 
negative ratings for students 7 attitudes towards achievement and 
26 percent in schools receiving negative ratings for parental support. 



4 Hill, P.T., Foster, G.E., & Gendler, T., High Schools with Character (Santa Monica, CA: The Rand 
Corporation, 1990). 
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Trends and Trouble Spots in Mathematics Instruction 



Although the two-year period between 1990 and 1992 is too short to 
establish trends, some signs of movement were noted toward reform in 
school mathematics. These changes in instructional context tend to support 
the gains in average achievement noted across the same time period. 

• Students reported taking more advanced coursework. At grade 8, the 
percentages of students taking pre-algebra increased (from 20 to 

28 percent) while the percentages enrolled in a general eighth-grade 
mathematics class decreased (from 61 to 49 percent). Also, there 
was a decrease from 18 to 14 percent between 1990 and 1992 in the 
percentage of twelfth graders reporting three or fewer mathematics 
courses during high school. 

• Teachers reported moving toward a more broadly based curriculum. 
Although most students (90 percent at grade 4 and 76 percent at 
grade 8) were receiving heavy emphasis in numbers and operations, 
there were increases between 1990 and 1992 in the percentages of 
fourth graders receiving moderate emphasis in geometry and 
introductory algebra concepts, and more eighth graders were 
receiving at least a moderate emphasis in measurement and geometry. 

• Students and teachers reported more access to and use of calculators and 
computers. In 1992, more fourth graders had access to school- 
owned calculators than in 1990 (59 compared to 44 percent). More 
eighth graders were permitted unrestricted classroom use of 
calculators, both generally and in testing situations. There were 
increases in computer access and use at grade 4. 

• Students were doing more daily problem solving from their textbooks. 
Teachers at grades 4 and 8 as well as students at all three grades 
reported an increase in daily problem solving from textbooks. In 
1992, teachers reported that 76 percent of the fourth graders and 
83 percent of the eighth graders solved problems from textbooks 
on a daily basis. As a related finding, at grade 8, teachers reported 
assigning somewhat more mathematics homework in 1992 than in 
1990, with fewer students (28 compared to 41 percent) given only 
15 minutes of mathematics homework per night. 

• Students reported more positive attitudes about the value of mathematics. 
More students in 1992 than in 1990 reported understanding the 
utility of mathematics to solve everyday problems and as part of 
job-related skills. 
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Stili, not all the reports from teachers and students can be viewed 
positively. 

• Teachers reported discrepancies in resource availability and expectations 
between top- and bottom-performing schools. Teachers reported that 
more students in bottom- than in top-performing one-third schools 
were in classrooms with only some or none of the necessary 
resources — 40 compared to 29 percent at grade 4, and 42 compared 
to 28 percent at grade 8. In 1992, 25 percent of the eighth graders in 
top one-third schools were expected to do 45 minutes of 
mathematics homework per night, compared to 16 percent in 
bottom one-third schools. The rather pervasive increase in calculator 
use at grade 8 was not observed in bottom-third schools. 



"DAT A.F ! 1 F 



Percentages of Fourth- and Eighth-Grade Students 
in Schools with only Some or None of the 
Necessary Resources 

Top One-Third Performing Schools 

Grade 4 I ■^■^Bf 29% 
Grade 8 | ^^^^^|28% 

Bottom One-Third Performing Schools 



Grade 4 
Grade 8 
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• Teachers and students reported very little change in the frequency with 
which students were asked to engage in extended problem-solving activity. 
Teachers reported no increases between 1990 and 1992 in the 
percentages of students receiving heavy instructional emphasis in 
reasoning and communication for either grade 4 or grade 8. Only 
about half the students or fewer were receiving heavy instructional 
emphasis in these two areas emphasized in the NCTM Standards. 5 
Teachers also reported that about half their students were never or 
hardly ever assessed using projects, portfolios, or presentations. 
The percentage of students reporting that they were never or hardly 
ever asked to write reports or do mathematics projects increased 
significantly between 1990 and 1992 (from 70 to 77 percent at grade 
8 and from 71 to 82 percent at grade 12). As a related finding, 

62 percent of the grade 8 students and 68 percent at grade 12 
reported that they never or hardly ever were asked to write a few 
sentences about how they solved a mathematics problem. 

• Despite a direct and powerful relationship between taking advanced 
mathematics courses and higher achievement, students reported an 
extremely low degree of mathematics coursework. About half of the 
eighth graders (49 percent) were taking eighth-grade mathematics, 
while those with higher average proficiency were enrolled in pre- 
algebra (28 percent) or algebra (20 percent). Eighth graders planning 
to proceed to more advanced coursework (geometry or algebra I) 

in ninth grade also had higher average proficiency. However, 
21 percent did not know what mathematics course they would 
take in grade 9. 

Only 42 percent of the twelfth graders reported taking eight 
semesters of mathematics coursework during their high school 
years. Twenty-three percent of the twelfth graders (36 percent in 
bottom-third schools) reported never studying geometry, which has 
become a "gatekeeper" course for access to higher education since 
most colleges are requiring this course prior to entrance. There were 
large discrepancies in the amounts of mathematics coursework 
reported among various subpopulations. 



5 National Council of Teachers of Mathematics, Curriculum and Evaluation Standards for School 
Mathematics (Reston, VA: National Council of Teachers of Mathematics, 1989). 
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Another Perspective on School Effectiveness 

In addition to examining the characteristics of schools with the highest 
mathematics achievement, it is also informative to examine factors 
associated with schools that are most effective in maximizing learning 
beyond the students' home background and the socioeconomic levels of 
the communities in which the schools are located. From a value-added 
perspective, schools with a more difficult educational task can be just as 
effective as those with fewer hurdles to overcome, and NAEP used 
hierarchical analysis methods to examine this issue. Unfortunately, the 
state of the art in educational measurement is not nearly sophisticated 
enough to provide definitive answers or to completely untangle the 
relationships among the myriad inputs to students' learning. No matter 
how sophisticated the approach, the adjustments can never be complete, 
and such is the case in attempts to examine the effects of socioeconomic 
status. (Also, please note that the teacher questionnaire data were not 
included in the analyses for technical reasons.) 

Nevertheless, two different analyses were conducted after average 
mathematics achievement was adjusted for variations in students' home 
background and the socioeconomic levels of the communities in which their 
schools were located. In one of these analyses, achievement was predicted 
based on school socioeconomic indicators. Schools with higher performance 
than their counterparts with similar socioeconomic characteristics were 
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considered to be the most effective, and schools that did not perform as well 
were considered to be the least effective. An analysis of differences between 
the 15 percent most and least effective schools according to the NAEP 
background data showed that in tl?<5 most effective schools, especially at the 
upper grades: 

• students watched less television 

• students were tested a moderate amount in mathematics class 

• students were taking advanced mathematics courses 

• students and parents had more positive attitudes toward 
academic achievement 

• there were fewer school problems 

• students read more for schoolwork each day (either at school or 
at home) 

• students worked mathematics problems from textbooks 
more frequently 

• more students went on to four-year colleges or universities 

In another analysis, variables and sets of variables were examined tc 
determine the major predictors of school effectiveness, with the following 
results: 

• Important classroom factors included moderate testing in mathematics 
class, doing problems from textbooks more frequently, and more 
frequent calculator use. 

• Important school characteristics were a positive school climate and 
limited problems. A stable student body, as opposed to a transient 
one, also was important. 

• The largest effects were related to having more students taking more 
advanced courses. The number of advanced mathematics courses 
taken was the most powerful predictor of students' mathematics 
performance after adjusting for variations in home background. 

• School-level socioeconomic indicators, such as community type 
and percentage of students participating in the subsidized lunch 
program, also were powerful predictors and were correlated with 
classroom, school, and course-taking factors. 
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Background Questionnaires 



The background-data collection effort associated with NAEP's 1992 
mathematics assessment involved asking students, their mathematics teachers, 
and the school principals to complete questionnaires. Students at all three 
grades completed questionnaires about demographic characteristics and 
their educational experiences in mathematics, including instructional 
activities, courses taken, use of specialized resources such as calculators, 
and their views about the value of the subject matter. 

The teachers of the fourth and eighth graders who participated in the 
assessment provided information about their own background and training. 
The fourth and eighth graders' teachers also provided information, 
classroom by classroom, about students' ability levels, instructional time, 
homework assignments, frequency of instructional activities, and the 
emphasis given to various mathematics content areas. 

School administrators at all three grades completed questionnaires 
about a variety of school characteristics, including absenteeism, policies 
about tracking, special priorities and school-wide programs, availability of 
resources, and school-wide problems. The student, teacher, and school 
background questionnaires were completed both as part of the national and 
Trial State Assessments. 

In formulating the extensive set of background questions associated 
with NAEP's 1992 mathematics assessment, three goals were kept in mind: 

• to provide an educational context for understanding data on 
student achievement 

• to identify differences in access to instruction and distribution of 
services for various types of students 

• to track changes in policy-relevant variables across time 

The 1992 NAEP Background Questionnaire Framework focused on five 
major educational policy areas. 6 These areas arc broad, encompassing 
domains inclusive enough to address the needs, interests, and concerns of a 
variety of NAEP audiences. The five areas included: instructional content, 
instructional practices and experiences, teacher characteristics, school 
conditions and context, and conditions outside of school that affect learning 
and instruction. 



* National Assessment of Educational Progress, 1992 Background Questionnaire Framework (Princeton, NJ: 
National Assessment of Educational Progress, Educational Testing Service, 1992). 
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Procedures and Methods 



As with all NAEP assessments; the schools and students participating in the 
1990 and 1992 mathematics assessments were selected through scientifically 
designed stratified random sampling procedures. Approximately 26,000 
fourth, eighth, and twelfth graders in 1,500 public and private schools across 
the country participated in the national assessment. For each jurisdiction 
participating in the Trial State Assessment Program, approximately 2,500 
students were sampled from approximately 100 public schools for each 
grade. Thus, a total of approximately 220,000 fourth- and eighth-grade 
students attending nearly 9,000 public schools participated in the 1992 Trial 
State Assessments. The *4 participating jurisdictions were: 



Alabama 


Louisiana 


Ohio 


Arizona 


Maine 


Oklahoma 


Arkansas 


Maryland 


Pennsylvania 


California 


Massachusetts 


Rhode Island 


Colorado 


Michigan 


South Carolina 


Connecticut 


Minnesota 


Tennessee 


Delaware 


Mississippi 


Texas 


District of Columbia 


Missouri 


Utah 


Florida 


Nebraska 


Virginia 


Georgia 


New Hampshire 


West Virginia 


Hawaii 


New Jersey 


Wisconsin 


Idaho 


New Mexico 


Wyoming 


Indiana 


New York 




Iowa 


North Carolina 


Guam 


Kentucky 


North Dakota 


Virgin Islands* 



♦Hie Virgin Islands participated in the testing portion of the 1992 Trial State Assessment Program. 
However, in accordance with the legislation providing for participants to review and give permission 
for release of their results, the Virgin Islands chose not to release their results at grade 4 in the 1992 
NAEP reports. 

Trend results from a comparable assessment conducted in 1990 are 
available for the nation and for the 37 states and territories that participated 
in both the 1990 and 1992 programs at grade 8. NAEP's Trial State Assessment 
Program began in 1990 with mathematics at grade 8 and expanded in 1992 
to include both grades 4 and 8, as well as reading at grade 4. The states 
that do not have trend data at grade 8 include: Maine, Massachusetts, 
Mississippi, Missouri, South Carolina, Tennessee, and Utah. 
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All NAEP data are collected by trained administrators. Data for the 
national assessment were collected by a field staff managed by Westat, Inc. 
However, in accordance with the NAEP legislation, data collection for the 
Trial State Assessment Program was the responsibility of each participating 
jurisdiction. Uniformity of procedures across states was achieved through 
training and quality control monitoring by Westat, Inc. Westat staff trained 
nearly 10,000 state assessment administrators using a video presentation 
accompanied by a scripted trainer's guide and practice exercises. Quality 
control was provided by unar^ounced, random monitoring of half the 
sessions in each state. The re ts of the monitoring indicated a high degree 
of quality and uniformity across sessions. 

The materials, including approximately 4 million written responses 
constructed by students in 1992, were scored by National Computer 
Systems in Iowa City, Iowa, using scoring rubrics developed by the NAEP 
Mathematics Test Development Committee and staff at Educational Testing 
Service (ETS). The scoring rubrics were developed prior to the assessment, 
revised on the basis of field-test results, and modified a final time following 
an examination of samples of student responses obtained in the actual 
assessment. To evaluate the scoring reliability, 25 percent of the papers for 
each question were scored by two different scorers. The percentage of exact 
agreement, averaged across the questions and papers, was 94 percent. 

ETS analyzed the assessment results to determine the percentage of 
students responding correctly to each multiple-choice and regular 
constructed-response question and the percentage of students responding in 
each of the categories for extended problem-solving tasks. Iterr response 
theory (IRT) methods were used to summarize results for each of the 
mathematics content areas, which included Numbers and Operations; 
Measurement; Geometry; Data Analysis, Statistics, and Probability; and 
Algebra and Functions. New for the 1992 NAEP assessment, a partial-credit 
scaling procedure employing a specialized IRT method was used to account 
for students' responses scored according to the five-point scoring guides 
used with the extended problem-solving tasks. An overall composite scale 
was developed by weighting each content area according to its importance 
in the framework. 7 Average proficiency on the composite scale, which 
ranges from 0 to 500, is the statistic primarily used in this report to compare 
overall mathematics performance for various groups of students. Unless 



7 Mathematics Objectives, 1990 Assessment (Princeton, NJ: National Assessment of Educational Progress, 
Educational Testing Service, 1988). 



33 

36 



otherwise noted, all changes or differences discussed in this report 
are statistically significant at the .05 level of significance. This means 
that the observed differences are unlikely to be due to chance or to 
sampling variability. 

Throughout the development and conduct of the 1992 assessment, 
the National Center for Education Statistics (NCES) and its contractors 
worked closely with the Trial State Assessment NETWORK, which 
includes representatives from all interested states. Federal funding 
permitted state education personnel to meet with staff members from 
NCES, the contractors, the National Assessment Governing Board (NAGB), 
and the Council of Chief State School Officers (CCSSO) at NETWORK 
meetings regularly held to review NAEP materials and procedures. Further 
details about the methods and procedures used in NAEP's 1992 mathematics 
assessment of the nation and states are provided in the Technical Report of 
the 1992 Trial State Assessment in Mathematics and The NAEP 1992 Technical 
Report (forthcoming). 

A Note on Interpretations 

The NAEP background questionnaires make it possible to examine the 
relationships between student proficiency and a wide variety of background 
factors, usually by relating performance to one or several variables at a time. 
The selection of background questions included in the NAEP mathematics 
assessment was guided by the NAEP 1992 Background Questionnaire 
Framework, derived by considering the wide body of available research 
about factors influencing student learning and the particular purposes 
of NAEP data collection. Because of their basis in research, the NAEP 
survey results often help to confirm our understanding of how school 
and home factors relate to achievement. Although the effects of schooling 
and instruction are of prime concern, these analyses do not reveal the 
underlying causes of the relationships between background factors and 
performance. The NAEP assessment results are most useful when they are 
considered in light of other knowledge about the education system, such as 
trends in instructional reform, changes in the school-age population, and 
societal demands and expectations. 
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This synthesis of findings from NAEP's 1992 Mathematics Assessments was 
adapted from the following NAEP reports: 

Mullis, I.V.S., Dossey, J.A., Owen, RH., & Phillips, G.W., NAEP 1992 
Mathematics Report Card for the Nation and States (Wash ngton, DC: National 
Center for Education Statistics, 1993). 

Dossey, J.A., Mullis, I.V.S., & Jones, CO., Can Students Do Mathematical 
Problem Solving? (Washington, DC: National Center for Education 
Statistics, 1993). 

Dossey, J.A., Mullis, I.V.S., & Gorman, S., How School Mathematics 
Functions (Washington, DC: National Center for Education Statistics, 1994). 

Mullis, I.V.S., Jenkins, R, & Johnson, E.G., Effective Schools in Mathematics 
(Washington, DC: National Center fcr Education Statistics, 1994). 
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